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On  ON 


1 .  Research  Project  Status 


Objective 

The  objective  of  this  effort  was  to  study  organ  level  modeling  and  simulation  for  surgical 
applications.  The  point  to  the  proposed  research  was  to  explore  the  feasibility  of 
developing  open  source,  open  architecture  models  of  different  levels  of  granularity  and 
spatio-temporal  scale  for  a  project  that  has  been  labeled  the  Digital  Human  project.  While 
the  emphasis  on  this  program  was  on  how  the  simulations  developed  will  allow  for  the 
interconnections  between  individual  organ  simulations,  and  between  different  types  of 
physical  processes  within  a  given  organ,  the  tools  were  developed  around  a  specific  test 
bed  application:  the  construction  of  a  heart  model  for  simulation  of  heart  surgery. 


Approach 

The  focus  of  this  project  was  on  organ  level  modeling:  interconnection  of  organ  level 
models  and  interfacing  of  these  models  with  the  higher  and  lower  levels  in  a  hierarchical 
simulation  framework.  There  were  three  main  thrust  areas:  Definition  of  a  software  API 
for  open  source  development  of  surgical  simulators,  development  of  a  simulation 
framework,  and  development  of  a  heart  simulation  for  surgery  using  the  specified  API. 

A  series  of  standard  APIs  for  organ  level  models  for  surgical  simulations  were  developed. 
The  emphasis  of  this  effort  was  on  the  extensibility  of  the  simulation  framework  and  the 
generality  of  the  interfaces  allowing  components  built  by  different  groups  and  individuals 
to  plug  together  and  be  reused.  Furthermore,  there  was  special  emphasis  on  support  for 
different  types  and  levels  of  abstraction  for  different  models. 

As  part  of  the  big  picture,  the  issues  related  to  development  of  an  open  source,  open 
architecture  software  framework  for  general  medical  simulation,  was  also  studied. 
Particular  focus  was  on  how  the  developed  organ  level  modeling  APIs  would  fit  into  the 
framework  of  a  larger  simulation,  such  as  the  Digital  Human  initiative.  This  research 
effort  was  performed  in  collaboration  with  the  members  of  the  DARPA  funded  BioSpice 
group  at  Berkeley  during  the  development  of  API  specifications  to  ensure  that  these  two 
initially  independent  but  complementary  frameworks  were  compatible  in  the  long  term. 

The  test-bed  application  considered  during  the  open  source  modeling  and  simulation  API 
and  the  simulation  framework  was  a  heart  model  for  surgery  simulation.  The  envisioned 
simulation  included  models  for  basic  electromechanical,  circulatory,  and  physiological 
behavior  of  the  heart,  to  evaluate  the  critical  aspects  of  the  design  decisions  for  the  API 
and  the  simulation  framework. 
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A  ccomplishmen  ts 

The  planned  progress  of  the  research  is  in  agreement  with  the  statement  of  work  proposed 
in  the  initial  white  paper.  Below  is  a  summary  of  the  accomplishments  of  this  effort.  A 
more  detailed  discussion  on  the  framework  and  the  APIs  can  be  found  in  the  appended 
document:  “GiPSi:  An  Open  Source/Open  Architecture  Software  Development 
Framework  for  Surgical  Simulation”,  M.  C.  Cavusoglu,  T.  G.  Goktekin,  F.  Tendick,  S. 
Sastry. 


•  Task  1:  API  Design 

The  developed  framework  provides  three  major  APIs:  I)  A  Simulation  API  for 
developing  reusable  models  of  organs,  2)  An  Interfacing  API  for  interfacing  dynamic 
models  defined  over  spatial  domains,  3)  A  Visualization  API  for  facilitating  the 
display  of  the  geometries  defined  in  the  models.  The  majority  of  the  models  in  organ 
level  simulations  involve  solving  multiple  time  varying  Partial  Differential  Equations 
(PDEs)  that  are  defined  over  spatial  domains  and  are  coupled  via  boundary 
conditions.  These  models  of  organs  and  physical  processes  associated  with  them  are 
represented  as  Simulation  Objects.  These  objects  define  the  basic  API  for  simulation, 
interfacing  and  visualization. 

1.  Simulation  API: 

Simulation  of  an  organ  model  defined  by  a  system  of  PDEs  first  requires  the 
discretization  of  the  domain  it  is  defined  on.  Therefore  the  API  that  defines  an 
organ  model  should  contain  a  proper  geometry  that  defines  its  discretized  domain, 
called  the  Domain  Geometry.  The  Simulation  Object  API  defines  a  set  of 
geometric  primitives  that  include  polygonal  surface  and  polyhedral  volume 
meshes.  After  discretization,  a  method  for  solving  the  PDE  system  should  be 
employed.  There  are  many  different  methodologies  for  solving  system  of  PDEs. 
This  effort  focused  on  providing  the  most  widely  used  ones.  Therefore,  APIs  for 
general  purpose  Mass  Spring  Damper  (MSD)  objects,  Einite  Element  Method 
(EEM)  objects  and  Einite  Difference  Method  (EDM)  objects  have  been 
developed. 

2,  Interfacing  API: 

Design  of  a  general  and  flexible  API  for  interfacing  multiple  organ  models  is 
critical  for  facilitating  shared  development  and  reuse.  Providing  such  an  API  has 
been  one  of  the  most  challenging  tasks  of  this  effort.  The  most  fundamental 
communication  between  two  arbitrary  organ  models  is  defined  via  the  boundary 
conditions  between  them.  The  API  first  provides  a  standard  definition  of  the 
boundary  of  an  object,  called  Boundary  Geometry.  Then  it  defines  a  set  of 
methods  to  set  and  get  values  on  the  boundary.  In  order  to  facilitate  the 
interfacing  of  two  arbitrary  models,  the  boundary  geometry  is  limited  only  to  a 
polygonal  surface  geometry.  However,  even  though  the  type  of  the  boundary 
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their  semantics  are  up  to  the  modeler  and  should  be  well  documented.  A  more 
general  information  passing  is  provided  as  well  by  a  Get/Set  mechanism,  i.e.  an 
object  can  read  and  write  values  inside  another  object  by  simply  using  Get(value) 
and  Set(value)  methods  provided  by  the  object  respectively.  The  set  of  values 
that  can  be  get  and  set  by  other  objects  and  their  semantics  are  again  left  to  the 
modeler.  Both  of  the  interfacing  methods  are  achieved  by  the  use  of  the 
Connector  Classes.  Since  the  connection  of  two  arbitrary  models  is  application 
dependent,  it  is  the  modeler's  task  to  develop  these  connectors. 

3.  Visualization  API: 

In  order  to  display  an  object  the  API  defines  another  geometry  dedicated  for 
visualization,  called  the  Display  Geometry.  Each  display  geometry  has  a  Display 
Manager  associated  with  it.  Display  managers  convert  the  data  in  geometries  into 
a  standard  format  used  by  the  visualization  module  where  the  actual  display  takes 
place.  This  makes  the  development  of  visualization  tools  and  development  of 
models  mutually  exclusive  and  ensures  the  modularity  and  the  flexibility  of  the 
system. 


•  Task  2:  Simulation  Framework 

The  main  focus  of  this  effort  was  to  design  an  open  source/open  architecture 
framework  for  developing  organ  level  simulations.  The  goal  was  to  facilitate  shared 
development  of  reusable  models,  to  accommodate  heterogeneous  models  of 
computation  (e.g  differential  equations  and  finite  state  machines),  and  to  provide  a 
framework  for  interfacing  multiple  heterogeneous  models  (e.g.  solid  mechanics,  fluid 
mechanics  and  bioelectricity).  Designs  of  basic  I/O  interfaces  for  visualization  for 
real-time  interactive  applications  are  also  included. 

The  overall  system  architecture  of  the  proposed  framework  is  shown  in  Fig.  1.  The 
models  of  physical  processes  such  as  muscle  mechanics  of  the  heart  are  represented 
as  Simulation  Objects  as  explained  in  Taskl.  Each  simulation  object  can  be  derived 
from  a  specific  computational  model  contained  in  Modeling  Tools  such  as  finite 
elements,  finite  differences,  lumped  elements  etc.  The  Computational  Tools  provide  a 
library  of  numerical  methods  for  low  level  computation  of  the  object's  dynamics.  The 
objects  are  created  and  maintained  by  the  Simulation  Kernel  which  arbitrates  their 
communication  to  other  objects  and  components  of  the  system.  I/O  subsystem 
provides  basic  user  input  through  the  haptic  interface  tools  and  basic  output  through 
visualization  tools.  There  are  also  Auxiliary  Functions  that  provide  application 
dependent  support  to  the  system  such  as  collision  detection  and  collision  response 
tools  that  are  widely  used  in  interactive  applications. 


3 


GiPSi  Simulator  Architecture 


Figure  1,  The  system  architecture 


Complete  implementation  of  this  extensive  framework  requires  time,  man  power  and 
eollaboration  between  different  groups  of  expertise.  One  of  the  goals  of  this  effort  is 
to  provide  tools  for  the  construction  of  a  heart  model  for  simulation  of  heart  surgery. 
Therefore  the  subset  of  the  framework  that  is  necessary  for  achieving  this  goal  is 
implemented; 

1,  Modeling  Tools: 

The  muscles  of  the  heart  and  the  blood  it  pumps  are  the  main  components  of  a 
heart  simulation  which  can  be  modeled  as  solid  and  fluid  mechanics  models 
respectively.  The  most  basic  modeling  tool  for  solid  mechanics  is  Mass  Spring 
Damper  (MSD)  systems.  Therefore,  first  a  general  purpose  MSD  object  has  been 
implemented.  However,  MSD  systems  have  limited  usage.  A  more  robust  and 
widely  accepted  tool  for  modeling  both  solid  and  fluid  mechanics  is  the  Finite 
Element  (FEM)  systems.  This  framework  implements  a  3D  FEM  model  for  solid 
objects  and  a  2D  FEM  model  for  fluid  objects.  Imposing  accurate 
incompressibility  on  3D  unstructured  tetrahedral  meshes  turned  out  to  be  very 
unstable  for  practical  linear  element  types  such  as  linear  velocity-constant 
pressure  and  linear  velocity-linear  pressure.  Therefore  the  implementation  a  3D 
FEM  model  for  fluid  objects  could  not  be  completed.  Instead,  a  3D  Finite 
Difference  Method  (EDM)  based  fluid  simulator  with  a  staggered  grid 
arrangement  has  been  implemented. 
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2,  Computational  Tools: 

In  order  to  solve  the  time  varying  PDE  systems  a  set  of  explicit  and  implicit  time 
integrators  have  been  implemented.  In  order  to  exploit  the  sparseness  of  the 
systems  resulting  from  the  discretization  of  the  fluid  and  elastic  models,  a  general 
purpose  Element-By-Element  Conjugate  Gradient  (CG)  solver  is  developed. 

3.  Visualization  Tools: 

Visualization  of  an  object  involves  displaying  the  geometry  of  the  object  on  the 
screen.  The  implementation  provided  uses  OpenGE  for  display.  The  geometry  to 
be  displayed  is  defined  in  the  simulation  object  as  discussed  before.  However,  to 
assure  modularity,  the  object  converts  its  geometry  data  into  a  standard  form 
using  the  display  manager  associated  with  the  type  of  geometry  it  has.  Then  the 
visualization  tool  accesses  this  data  through  the  object  pool  maintained  by  the 
simulation  kernel  and  displays  it.  The  standard  format  used  is  simply  the  list  of 
vertex  positions,  vertex  normals,  vertex  colors  and  connectivity  information. 


•  Task  3:  Heart  Model  for  Surgical  Simulation 

The  objective  of  this  task  was  to  develop  a  heart  model  for  surgical  simulation  to 
evaluate  the  API  design.  Development  of  a  complete  heart  model  with  detailed 
mechanical,  electrical,  and  fluid  models  coupled  was  planned  in  the  original  proposal. 
However,  major  difficulties  were  encountered  during  the  development.  Eirst,  the  low 
order  EEM  based  3D  incompressible  fluid  model  proved  to  be  very  unstable  on 
unstructured  meshes  that  are  required  to  define  the  complex  geometry  of  a  heart 
object.  Therefore,  an  EDM  based  model  has  been  implemented.  However,  these 
models  simulate  the  entire  fixed  grid  that  the  fluids  lives  in  and  are  difficult  to 
interface  with  arbitrary  MSD  or  EEM  based  solid  models.  Therefore,  instead  of 
developing  a  single  large  scale  model,  we  have  developed  smaller  scale  models  to  test 
individual  parts  of  the  API. 
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APPENDIX  A 


GiPSi:  An  Open  Source/Open  Architecture 
Software  Development  Framework 
for  Surgical  Simulation 


Tolga  G.  Goktekin^,  M.  Genk  (Javu§oglu^, 

Frank  Tendick^,  and  Shankar  Sastry^ 

^  Dept,  of  Electrical  Eng.  and  Computer  Sci.,  Case  Western  Reserve  University 
^  Dept,  of  Electrical  Eng.  and  Computer  Sci.,  University  of  California,  Berkeley 
®  Dept,  of  Surgery,  University  of  California,  San  Francisco 


Abstract.  In  this  paper  we  propose  an  open  source/open  architecture 
framework  for  developing  organ  level  surgical  simulations.  Our  goal  is 
to  facilitate  shared  development  of  reusable  models,  to  accommodate 
heterogeneous  models  of  computation,  and  to  provide  a  framework  for 
interfacing  multiple  heterogeneous  models.  The  framework  provides  an 
intuitive  API  for  interfacing  dynamic  models  defined  over  spatial  do¬ 
mains.  It  is  specifically  designed  to  be  independent  of  the  specifics  of  the 
modeling  methods  used  and  therefore  facilitates  seamless  integration  of 
heterogeneous  models  and  processes.  Furthermore,  each  model  has  sep¬ 
arate  geometries  for  visualization,  simulation,  and  interfacing,  allowing 
the  modeler  to  choose  the  most  natural  geometric  representation  for  each 
case.  I/O  interfaces  for  visualization  and  haptics  for  real-time  interactive 
applications  have  also  been  provided. 

1  Introduction 

Gomputer  simulations  have  become  an  important  tool  for  medical  applications, 
such  as  surgical  training,  pre-operative  planning,  and  biomedical  research.  How¬ 
ever,  the  current  state  of  the  field  of  medical  simulation  is  characterized  by  scat¬ 
tered  research  projects  using  a  variety  of  models  that  are  neither  inter-operable 
nor  independently  verifiable  models.  Individual  simulators  are  frequently  built 
from  scratch  by  individual  research  groups  without  input  and  validation  from 
a  larger  community.  The  challenge  of  developing  useful  medical  simulations  is 
often  too  great  for  any  individual  group  since  expertise  is  required  from  differ¬ 
ent  fields.  The  motivation  behind  this  study  is  our  prior  experience  in  surgical 
training  simulators  and  physically  based  modeling  [10, 11]. 

The  open  source,  open  architecture  software  development  model  provides  an 
attractive  framework  to  address  the  needs  of  interfacing  models  from  multiple 
research  groups  and  the  ability  to  critically  examine  and  validate  quantitative 
biological  simulations.  Open  source  models  ensure  quality  control,  evaluation, 
and  peer  review,  which  are  critical  for  basic  scientific  methodology.  Further¬ 
more,  since  subsequent  users  of  the  models  and  the  software  code  have  access 
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to  the  original  code,  this  also  improves  the  reusability  of  the  models  and  inter- 
connectibility  of  the  software  modules.  On  the  other  hand,  an  open  architecture 
simulation  framework  allows  open  source  or  proprietary  third  party  development 
of  additional  models,  model  data,  and  analysis  and  computation  modules. 

In  this  paper  we  propose  GiPSi  (General  Interactive  Physical  Simulation 
Interface),  an  open  source/open  architecture  framework  for  developing  surgical 
simulations  such  as  interactive  surgical  training  and  planning  systems.  The  main 
goal  of  this  framework  is  to  facilitate  shared  model  development  and  simulation 
of  organ  level  processes  as  well  as  data  sharing  among  multiple  research  groups. 
To  address  these,  we  focused  on  providing  support  for  heterogeneous  models  of 
computation  (e.g.  differential  equations,  finite  state  machines  and  hybrid  sys¬ 
tems)  and  defined  APIs  for  interfacing  various  heterogeneous  physical  processes 
(e.g.  solid  mechanics,  fluid  mechanics  and  bioelectricity).  In  addition,  I/O  in¬ 
terfaces  for  visualization  and  haptics  for  real-time  interactive  applications  have 
been  provided.  The  implementation  of  the  framework  is  done  using  G-l— I-  and  it 
is  platform  independent. 

An  important  difference  of  GiPSi  from  earlier  object-oriented  tools  and  lan¬ 
guages  for  modeling  and  simulation  of  complex  physical  systems,  such  as  Mod- 
elica  [8],  Matlab  Simulink  [7],  and  Ptolemy  [3],  is  its  focus  on  representing  and 
enforcing  time  dependent  spatial  relationships  between  objects,  especially  in  the 
form  of  boundary  conditions  between  interfaced  and  interacting  objects.  The 
APIs  in  GiPSi  are  also  being  designed  with  a  special  emphasis  on  being  general 
and  independent  of  the  specifics  of  the  implemented  modeling  methods,  unlike 
earlier  dynamic  modeling  frameworks  such  as  SPRING  [9]  or  AlaDyn-3D  [6], 
where  the  underlying  models  used  in  these  physical  modeling  tools  are  woven 
into  the  specifications  of  the  overall  frameworks  developed.  This  allows  GiPSi  to 
seamlessly  integrate  heterogeneous  models  and  processes,  which  is  not  possible 
with  the  earlier  dynamic  modeling  frameworks  [5] . 

2  Overview 

The  goal  of  GiPSi  is  to  provide  a  framework  that  facilitates  shared  develop¬ 
ment  that  would  encourage  the  extensibility  of  the  simulation  framework  and 
the  generality  of  the  interfaces  allowing  components  built  by  different  groups  and 
individuals  to  plug  together  and  reused.  Therefore,  modularity  through  encapsu¬ 
lation  and  data  hiding  between  the  components  should  be  enforced.  In  addition, 
a  standard  interfacing  API  facilitating  communication  among  these  components 
needs  to  be  provided. 

We  are  developing  our  tools  on  a  specific  test-bed  application:  the  construc¬ 
tion  of  a  heart  model  for  simulation  of  heart  surgery.  This  test-bed  model  cap¬ 
tures  the  most  important  aspects  of  the  general  problem  we  are  trying  to  address: 
i)  multiple  heterogeneous  processes  that  need  to  be  modeled  and  interfaced,  and 
a)  different  levels  of  abstraction  possible  for  the  different  processes.  In  the  heart 
surgery  simulation,  several  different  processes,  namely  physiology,  bioelectrical 
activity,  muscle  mechanics,  and  blood  dynamics,  need  to  modeled.  Physiological 
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processes  regulate  the  bioelectrical  activity,  which,  in  turn,  drives  the  mechanical 
activity  of  the  heart  muscle.  Muscle  dynamics,  coupled  with  the  fluid  dynam¬ 
ics  of  the  blood,  determine  the  resulting  motion  of  the  heart  [2].  Models  for  all 
these  processes  need  to  be  intimately  coupled:  the  mechanical  and  fluid  models 
through  a  boundary  interaction,  and  the  electrochemical  and  mechanical  models 
through  a  volume  interaction. 

The  overall  system  architecture  of  GiPSi  is  shown  in  Fig.  1.  The  models  of 
physical  processes  such  as  muscle  mechanics  of  the  heart  are  represented  as  Sim¬ 
ulation  Objects  (Sect.  3).  Each  simulation  object  can  be  derived  from  a  specific 
computational  model  contained  in  Modeling  Tools  such  as  finite  elements,  finite 
differences,  lumped  elements  etc.  The  Computational  Tools  provide  a  library  of 
numerical  methods  for  low  level  computation  of  the  object’s  dynamics.  These 
tools  include  explicit/implicit  ordinary  differential  equation  (ODE)  solvers,  lin¬ 
ear  and  nonlinear  algebraic  system  solvers,  and  linear  algebra  support.  The  ob¬ 
jects  are  created  and  maintained  by  the  Simulation  Kernel  which  arbitrates  their 
communication  to  other  objects  and  components  of  the  system  (Sect.  6).  One 
such  component  is  the  I/O  subsystem  which  provides  basic  user  input  provided 
through  the  haptic  interface  tools  and  basic  output  through  visualization  tools 
(Sect.  4).  There  are  also  Auxiliary  Functions  that  provide  application  dependent 
support  to  the  system  such  as  collision  detection  and  collision  response  tools  that 
are  widely  used  in  interactive  applications  (Sect.  5) 


Fig.  1.  The  system  architecture  of  GiPSi 
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3  Simulation  Objects 

In  this  framework,  organs  and  physical  processes  associated  with  them  are  repre¬ 
sented  as  Simulation  Objects.  These  objects  define  the  basic  API  for  simulation, 
interfacing,  visualization  and  haptics  (see  Fig.  2a). 

Each  Simulation  Object  can  be  a  single  level  object  implementing  a  specific 
physical  process  or  can  be  an  aggregate  of  other  objects  creating  a  hierarchy  of 
models.  For  example,  if  we  were  interested  only  in  muscle  model  of  a  beating 
heart,  then  we  would  define  the  heart  as  a  single  object  that  simulates  the  muscle 
mechanics.  However,  if  we  were  to  model  a  more  sophisticated  heart  with  both 
muscle  and  blood  models,  then  our  heart  object  would  be  an  aggregate  of  two 
objects,  one  implementing  the  muscle  mechanics  and  the  other  implementing  the 
blood  dynamics.  The  specific  coupling  of  these  muscle  and  blood  objects  would 
be  implemented  at  their  aggregate  heart  object  (see  Fig.  2c). 


Class  SimulationObject  { 

Class 

Class 

Geometry  DisplayGeometry; 

FEMObject:SimulationObject  { 

MSDObject:SimulationObject  { 

Domain  DomainGeometry; 

FEMNode  *Nodes; 

MSDNode  *Nodes; 

Boundary  BoundaryGeometry; 

FEMEIement  ^Elements; 

MSDSpring  *Springs; 

SimulateQ; 

AssembleO; 

DisplayO; 

} 

HapticsO; 

} 

} 

Class  ElectrochemicahSimulationObject  { 

//  User  defined  custom  electrochemical  model 

} 

a) 


b) 


Class  HeartiSimulationObject  { 

Electrochemical  Bioelectricity; 

MSDObject  Muscle; 

FEMObject  Blood; 


c) 


Fig.  2.  a)  Simulation  Object,  b)  Examples  of  modeling  tool  and  user  defined  objects, 
c)  Heart  object 


The  majority  of  the  models  in  organ  level  simulations  involve  solving  mul¬ 
tiple  time  varying  PDEs  that  are  defined  over  spatial  domains  and  are  coupled 
via  boundary  conditions,  e.g.  a  structural  model  representing  the  heart  muscles 
coupled  with  a  fluid  model  representing  the  blood  which  share  the  inner  surface 
of  the  heart  wall  as  their  common  boundary.  Our  goal  is  to  design  a  flexible 
API  that  facilitates  the  shared  development  and  reuse  of  models  based  on  these 
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PDEs.  Therefore  the  focus  of  our  effort  is  to  provide:  i)  a  common  geometric 
representation  of  the  domain,  ii)  a,  library  of  tools  for  solving  these  PDEs,  Hi)  a 
standard  API  for  coupling  them. 

3.1  Simulation  API 

The  first  step  in  solving  a  continuous  PDE  is  to  discretize  the  spatial  domain 
it  is  defined  on.  Therefore,  every  object  must  contain  a  proper  geometry  that 
describes  its  discretized  domain,  called  the  Domain  Geometry.  The  definition 
of  this  geometry  is  flexible  enough  to  accommodate  the  traditional  mesh  based 
methods  as  well  as  point  based  (mesh  free)  formulations.  GiPSi  defines  a  set  of 
geometries  that  can  be  used  as  a  domain  including  but  not  limited  to  polygonal 
surface  and  polyhedral  volume  meshes.  In  our  current  implementation  we  provide 
geometries  for  triangular  and  tetrahedral  meshes. 

Second,  a  method  for  solving  a  PDE  should  be  employed  such  as  Finite 
Element  Methods  (FEM),  Finite  Difference  Methods  (EDM)  or  Mass-Spring- 
Damper  (MSD)  methods.  Basic  general  purpose  objects  that  implement  these 
methods  are  provided  as  Modeling  Tools,  e.g.  there  is  a  general  customizable 
FEM  object  that  implements  the  basics  of  the  finite  element  method  (see  Fig.  2b). 
For  example,  an  FEM  based  fluid  model  with  linear  elements  can  be  modeled 
as  an  FEM  object  with  a  tetrahedral  volume  mesh  as  its  Domain  Geometry  and 
with  Navier-Stokes  equations  as  its  user  defined  PDE.  So  far  we  have  imple¬ 
mented  objects  for  FEM  and  MSD  methods.  GiPSi  also  provides  a  library  of 
numerical  analysis  tools  in  the  Gomputational  Tools  that  can  be  used  to  solve 
these  discretized  equations.  Our  current  implementation  provides  explicit  and 
implicit  integrators,  some  popular  direct  and  iterative  linear  system  solvers  and 
G-l— I-  wrappers  around  a  subset  of  BLAS  and  LAPAGK  functions  [1]. 

3.2  Interfacing  API 

In  addition  to  representing  the  domain  geometry  and  assigning  a  method  of 
computation,  the  simulation  API  also  needs  to  provide  a  standard  means  to 
interface  multiple  objects.  In  the  models  mentioned  above,  the  basic  coupling 
of  two  objects  are  defined  via  the  boundary  conditions  between  them.  There¬ 
fore,  we  need  to  provide  an  API  to  facilitate  the  passing  of  boundary  conditions 
between  different  models.  First,  we  need  a  common  definition  of  the  boundary, 
i.e.  each  object  needs  to  have  a  specific  Boundary  Geometry.  In  our  current  im¬ 
plementation,  we  chose  triangular  surfaces  as  our  standard  boundary  geometry. 
Even  though  the  type  of  the  boundary  geometry  is  fixed  for  every  object,  the 
values  that  can  be  set  at  the  boundary  and  their  semantics  are  up  to  the  mod¬ 
eler  and  should  be  well  documented.  Moreover,  it  is  also  the  developer’s  task 
to  interface  two  objects  with  different  semantics  on  the  boundary.  For  example, 
a  generic  fluid  object  can  compute  velocities  and  pressures  on  its  boundary.  In 
order  to  interface  it  with  a  structural  object  that  requires  forces  on  its  boundary 
as  boundary  conditions,  the  developer  needs  to  convert  the  boundary  pressure 
values  to  boundary  forces  by  integrating  the  pressure  on  the  boundary. 
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Use  of  boundary  conditions  is  not  the  only  interfacing  scheme  for  objects.  For 
example,  the  coupling  between  the  electrochemical  and  mechanical  models 
(excitation-contraction  coupling)  in  the  heart  is  through  the  commonly  occupied 
volume  rather  than  a  shared  boundary.  A  more  general  information  passing  is 
provided  by  a  simple  Get/Set  scheme,  i.e.  an  object  can  read  and  write  values 
inside  another  object  by  simply  using  Get(value)  and  Set(value)  methods  pro¬ 
vided  by  the  object  respectively.  The  set  of  values  that  can  be  get  and  set  by 
other  objects  and  their  semantics  are  again  left  to  the  modeler.  In  the  above 
example,  the  electrochemical  model  sets  the  internal  force  values  of  the 
mechanical  model  based  on  the  excitation  level  which  in  turn  result  in  the 
contraction  of  the  muscles. 

Both  interfacing  through  a  surface  via  boundary  conditions  and  interfacing 
through  a  volume  (domain)  via  the  Get/Set  scheme  are  achieved  by  the  use  of 
the  Connector  classes.  Since  the  connection  of  two  arbitrary  models  is  application 
dependent,  it  is  the  modeler’s  task  to  develop  these  connectors.  Fig.  3  shows  two 
connector  classes  that  interface  three  basic  models  contained  in  the  aggregate 
Heart  model.  The  first  connector  class  provides  basic  communication  between 
the  Bioelectrical  and  Muscle  models  through  their  volumetric  domain.  It  basically 
jetsVne  excitation  levels  from  the  Bioelectric  models  (Domain  1),  converts  them  to 
stress  and  sets  the  stress  tensor  values  in  the  Muscle  model  (Domain  2).  The 
second  connector  interfaces  the  Lumped  Fluid  Blood  model  with  the  Muscle 
model  through  their  surfaces  via  boundary  conditions.  In  this  example  the 
communication  is  in  both  ways.  The  connector  class  reads  the  displacement 
values  on  the  Muscle  boundary  (Boundary  1),  converts  them  into  velocity  and 
passes  the  velocities  to  Fluid  model  (Boundary  2)  as  boundary  conditions.  Sim¬ 
ilarly  it  receives  the  boundary  pressure  values  from  Boundary  2,  converts  them 
into  forces  and  passes  them  to  Boundary  1  as  traction  values  on  the  boundary. 

3.3  Visualization  API 

In  order  to  display  an  object  we  again  need  a  geometry  dedicated  for 
visualization.  This  geometry  is  called  the  ^)isp[ay  geometry  and  can  be  of  any  type 
of  geometry  defined  in  GiPSi.  Each  display  geometry  has  a  ‘DispCay  Manager  asso- 
ciated  with  it.  Display  managers  convert  the  data  in  geometries  into  a  standard 
format  used  by  the  visualization  module  where  the  actual  display  takes  place 
(see  Sect.  4.2  for  details).  This  makes  the  development  of  visualization  tools  and 
development  of  models  mutually  exclusive  and  ensures  the  modularity  and  the 
flexibility  of  the  system. 

3.4  Haptics  API 

Haptic  interfacing  with  the  simulation  object  uses  the  multi-rate  simulation 
method  proposed  by  Qavu§oglo  in  [4].  In  this  method,  each  simulation  object  in 
haptic  interaction  provides  local  dynamic  and  geometric  models  for  the  haptic 
interface.  The  local  dynamic  model  is  a  low-order  linear  approximation  of  the  full 
deformable  object  model,  constructed  by  the  simulation  object  from  the  full 
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Fig.  3.  Connector  class  example 


model  at  its  update  intervals,  and  the  local  geometric  model  is  a  planar  approx¬ 
imation  of  the  local  geometry  of  the  simulation  object  at  the  haptic  interfacing 
location.  These  local  models  are  used  by  the  haptic  interface,  running  at  a  sig¬ 
nificantly  higher  update  rate  than  the  dynamic  simulations,  for  estimating  the 
inter-sample  interaction  forces  and  inter-sample  collisions. 

4  Input/Output  Subsystem 

The  Input/Output  subsystem  provides  basic  tools  for  interacting  with  the  ob¬ 
jects.  Currently,  GiPSi  provides  haptics  tools  for  input  and  visualization  tools 
for  output.  These  tools  provide  modularity  and  encapsulation  of  data,  and  define 
a  standard  API  for  model  developers. 


4.1  Haptics 

Haptic  interfaces  require  significantly  higher  update  rates,  usually  in  the  order 
of  1  kHz,  than  are  possible  for  the  rest  of  the  physical  models,  which  are  typi¬ 
cally  run  at  update  rates  in  the  order  of  10  Hz.  It  is  not  possible  to  increase  the 
update  rate  of  the  physical  models  to  the  haptic  rate  with  their  full  complexity 
due  to  computational  limitations,  or  to  decrease  the  haptic  update  rate  to  phys¬ 
ical  model  update  rates  due  to  stability  limitations.  As  described  in  section  3, 
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GiPSi  handles  this  conflicting  requirements  using  a  multi-rate  simulation  scheme 
[4].  The  Haptic  I/O  module  completely  encapsulates  the  haptic  interface  and  its 
real-time  update  rate  requirements,  and  provides  a  standard  API  for  all  of  the 
simulation  objects  which  will  be  haptically  interactive.  The  interface  between 
the  haptic  I/O  module  and  the  simulation  objects  is  through  the  local  dynamic 
and  geometric  models  provided  by  the  simulation  objects,  and  the  haptic  in¬ 
strument  location  and  interaction  forces  provided  by  the  haptic  I/O  module. 
The  instrument-object  interaction  forces  are  applied  to  the  objects  through  the 
object  boundary  conditions  and  the  instrument-object  collision  detections  are 
handled  no  differently  than  the  regular  object-object  collisions. 

4.2  Visualization 

Visualization  of  an  object  involves  displaying  the  geometry  of  the  object  on  the 
screen.  In  our  current  implementation  we  use  OpenGL  for  display.  The  geometry 
to  be  displayed  is  deflned  in  the  object  as  discussed  in  Sec. 3. 3.  However,  to 
assure  modularity,  the  object  converts  its  geometry  data  into  a  standard  form 
using  the  display  manager  associated  with  the  type  of  geometry  it  has.  Then 
the  visualization  tool  accesses  this  data  through  the  object  pool  maintained  by 
the  simulation  kernel  and  displays  it.  In  our  current  design,  the  standard  format 
used  is  simply  the  list  of  vertex  positions,  vertex  normals,  vertex  colors  and 
connectivity  information. 


5  Collision  Detection/Collision  Response 

In  interactive  surgical  simulations  one  needs  to  detect  collisions  to  prevent  pen¬ 
etration  between  objects  in  the  system,  such  as  organ  models  and  tools  used 
during  surgery.  Therefore  collision  detection  (GD)  and  collision  response  (GR) 
play  a  very  important  role.  In  our  framework,  GD  module  detects  the  collisions 
between  boundary  geometries  of  different  models  and  the  GR  module  computes 
the  required  response  to  resolve  these  collisions  in  terms  of  displacements  and/or 
penalty  forces  and  communicates  the  result  to  the  models  as  displacement  or 
force  based  boundary  conditions.  The  models  process  these  boundary  conditions 
if  necessary  and  iterate.  As  a  result,  the  mechanics  of  contact  detection  and 
resolution  becomes  transparent  to  the  model  developer. 

6  Simulation  Kernel 

The  simulation  kernel  acts  as  the  central  core  where  everything  above  comes 
together.  Its  tasks  include  the  management  of  the  top  level  object  pool,  coordi¬ 
nation  of  the  object  interactions,  and  arbitration  of  the  communication  between 
the  components.  The  part  which  coordinates  the  top  level  objects  is  provided  by 
the  user.  This  coordination  involves  specifying  the  execution  order  of  the  models 
and  the  specific  interfacing  between  them,  allowing  the  user  to  properly  inter¬ 
pret  the  semantics  of  the  individual  top  level  objects  and  the  interfacing  between 
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them,  based  on  the  specific  application  that  the  simulation  is  being  developed 
for. 


7  Conclusion 

We  have  presented  an  open  source/open  architecture  framework  for  organ  level 
simulations  that  facilitates  shared  development  and  reuse  of  models.  This  frame¬ 
work  provides  an  intuitive  API  for  interfacing  dynamic  models  defined  over  spa¬ 
tial  domains.  In  addition,  it  is  independent  of  the  specifics  of  modeling  methods 
and  thus  facilitates  seamless  integration  of  heterogeneous  models  and  processes. 
Furthermore,  each  model  has  separate  geometries  for  visualization,  simulation, 
and  interfacing.  This  lets  the  modeler  choose  the  most  natural  geometric  repre¬ 
sentation  for  each. 

We  want  to  emphasize  that  the  framework  proposed  in  this  paper  is  a  work 
in  progress.  It  is  intended  to  be  a  draft  that  will  be  modified  according  to  the 
feedback  we  receive  from  the  broader  surgical  simulation  community.  As  we  indi¬ 
cated  throughout  the  paper,  the  implementation  itself  is  incomplete  and  is  only 
presented  as  a  proof  of  concept.  If  the  framework  is  adopted,  the  implementation 
can  easily  be  extended  by  the  community.  Therefore,  we  plan  to  have  a  meeting 
with  the  interested  parties  at  ISMS  to  discuss  the  future  of  the  framework. 
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